MyCourses - Learning Management System

•

Course assignment for Interaction Design and Evaluation - https://noppabackup.aalto.fi/unrestricted/t-121.5450/esite/.

Team members: Cristian Grama, Magda Laszlo, Denis Laure, Judit Stiber, Hanna-Mari Kinnunen

The goal of the course is to be able to select suitable usability evaluation methods in various situations, and to argument your choices, as well as the findings of the evaluations. Furthermore, being able to design and carry out usability evaluations, and to construct a new user interface or a prototype suitable for user testing. The assignment was to conduct a usability evaluation for MyCourses - Aalto University's new learning management system.

The team analyzing usability test sessions

Usability Inspections

Heuristic Evaluation

We conducted the inspections based on the 1993 Nielsen’s heuristics:

1. Simple and natural dialogue

2. Speak the users’ language

3. Minimise users’ memory load

4. Consistency

5. Feedback
6. Clearly marked exits
7. Shortcuts
8. Good error messages

9. Prevent errors
10. Help and documentation

Cognitive Walkthrough

It is a task based inspection that focuses on the ease of learning from a learning by exploration viewpoint. The goal is to find problems to be fixed.

Back-of-the-envelope

Predictive models - Action analysis

List the steps required for a task

We identified a total of 79 issues.

Contextual interviews

Conducted two contexteual interviews of 1 hour with 2 professors of Aalto University concerning their usage of Noppa and Moodle.

Usability Test

Preparation

Creating a step-by-step document for the testing.

Creating a letter template to contact with professors.

Creating the scenarios and the tasks for the testing. We based the tasks and scenarios on our contextual interviews to test the system's most used functions by the teachers. We also refered back to our usability inspection and created tasks to see if teachers really have problems where we identified usability issues.

Creating a Pre-test Questionnaire.

Preparing and choosing pictures for AXE Evaluation.

Writing base questions for semi structured interview after the testing.

We ran a pre-pilot test with a computer science student to practice the process and fix obvious errors before testing with the teachers.

Test Sessions

We ran a pilot test where our course teacher was also invited. The main learning of the session was: never count on the video recording technique, it will let you down, take notes and have a back-up system, always. Then, we ran 6 more test sessions.

Each test session consisted of 7 tasks identified as basic tasks and 1 out of 3 advanced tasks. The actual testing of the system was limited to 30-35 minutes, while a total session lasted about 1 hour.

Analysis

Summarizing which tasks were the hardest to perform for the users - most mistakes. Furthermore, collecting the similar issues from different users, the more users ran into the same issue the more important the issue is.

The most important outcome of the usability was that we are not teachers, teachers got used to the system really fast and quite smooth which was unexpected to us.

5 out of the 7 most critical issues identified during the usability evaluations was confirmed on the usability test sessions

Prototype

Personal Sketches

After the analysis of the usability sessions each team member created some personal sketches about how to improve the MyCourses. These sketches mainly concentrated aroung the general organization of the class structure and the quizes. We identified that the inconsistent class structure is not a problem for the teachers, but it is a problem for the students. Teachers didn't struggle to create their content, and they didn't think much about how other teachers organize their content. Therefore, we decided to improve the quizes. An easier to use quiz builder could attract those teachers to use the build-in creator who find the current usage too complicated.

First prototype

We first conducted some research about the existing quiz creating softwares. Then we sketched out our UI idea. Finally, we implemented it using Axure.

The identified goals for designing the prototype:
- Create a lighter structure
- Design a minimalist, more intuitive UI
- Place focus on content creation
- Eliminate clutter, but reinforce essential functionalities
- Keep advanced functionalities, but hidden

Validation

We validated our prototype with 3 different type of teachers: 1 from our previous usability test, 1 who haven't seen the new system at all and 1 who already tried the new system, but not with us. These usability tests were conducted in the same condition as the first 7.

Some of the feedback:

“I am waiting to use it”

“I would rather use this quiz than the Moodle one”

Improved Prototype

Based on the usability tests, we improved our Axure prototype.

Final Prototype

Learnings

- Conducted Usability Studies: Back of the Envelop, Cognitive Walkthrough, Heuristic Evaluation
- Created a complete report on the usability issues
- Conducted Contextual Interviews with professors from Aalto University
- Designed and Conducted a total of 7 Usability Test Session with professors from Aalto University (including a Pilot Test Session)
- Analysed Usability Test Session results
- Created a Prototype based on the analysis
- Experimented with Axure prototyping software
- Designed and Conducted a total of 3 Usability Test Session with our prototype with professors from Aalto University